Image sensing apparatus, image sensing method, image sensing program, recording apparatus, recording method, and recording program

ABSTRACT

An image sensing apparatus and method utilizing an image sensor, an auto white balance adjustment section, a change point detection section, and an electronic mark data generation section. The image sensor senses an image of an object and outputs an image signal corresponding to the image of the object. The auto white balance adjustment section automatically adjusts white balance of the image signal corresponding to a change of the image signal. The change point detection section detects a change point of the white balance corresponding to at least an adjustment value of the white balance while the image sensor is sensing the image of the object. The electronic mark data generation section generates electronic mark data for adjusted white balance based on time information of the change point detected by the change point detection section.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2004-243265 filed in the Japanese Patent Office on Aug.24, 2004, the entire contents of which are hereby incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image sensing apparatus, an imagesensing method, an image sensing program, a recording apparatus, arecording method, and a recording program that allow both image signalsof objects shot by a video camera and operation information thereof tobe recorded.

2. Description of the Background Art

Image signals of objects shot by a video camera are recorded to arecording medium such as a video tape, a disc shaped recording medium,or a semiconductor memory. Thereafter, when necessary, the recordedimage signals are edited and used. In brief, the editing operation isperformed such that while the user is watching images reproduced asimage signals from the recording medium, he or she extracts desired cutsand marks a start point, referred to as an in-point, and an end point,referred to as an out-point, each of the extracted cuts. Correspondingto the marked in-points and out-points, the user connects a plurality ofcuts in his or her desired manner and records the connected images tofor example another recording medium. As a result, so-called completepackage images that are final images are obtained.

The user may need to extract his or her desired cuts, while he or she iswatching images. Thus, to create perfect package images, the user mayneed to extract one or several cuts from many images. Thus, the user mayhave to reproduce many images, spending a long time.

To solve such a problem, a system that extracts feature points fromimages, searches images for the feature points, and extracts desiredcuts has been proposed. As feature points, change points of operationstates of the video camera can be used. When operation states of thevideo camera, for example diaphragm, zoom, focus, white balance,amplifier gain, and shutter speed have changed, the object or theshooting environment likely changed or the user likely intentionallyoperated the video camera. Thus, these change points can be used asfeature points of images.

In particular, it is thought that a change point of an auto trackingwhite balance (ATW) that causes a white balance control to automaticallyfollow a surrounding color temperature corresponds to a large change ofa shooting environment such as a shooting location change from outdoorto indoor (or from indoor to outdoor), from room to room (e.g., from aroom with an incandescent lamp to a room with a fluorescent lamp). Thus,change points of the auto tracking white balance are very effective toextract cuts.

Japanese Patent Laid-Open Publication No. 9-198849 describes a structureof a video camera that obtains changes of its operation states when itshoots objects and records the obtained changes as camera operationinformation together with image signals to a recording medium. Accordingto JP 9-198849, when the user shoots objects, the video camera obtainscamera operation information and records the information together withimage signals to a recording medium frame by frame. When the user editsimages, he or she searches for them corresponding to the cameraoperation information frame by frame. With the structure described in JP9-198849, the user can quickly search his or her desired scenescorresponding to the camera operation states.

SUMMARY OF THE INVENTION

With the structure disclosed in JP 9-198849, since camera operationinformation is obtained and recorded frame by frame, the recordingcapacity of the recording medium needs to be increased. This leads to aproblem when many kinds of camera operation information are obtained andrecorded.

In addition, to search for scenes corresponding to camera operationinformation, it may be needed to determine whether concerned items ofcamera operation information exceed their threshold values. With thestructure disclosed in JP 9-198849, when the user edits images, he orshe may need to search camera operation information to determine whetherconcerned items of camera operation information exceed their thresholdvalues, spending a long processing time.

In view of the foregoing, it would be desirable to provide an imagesensing apparatus, an image sensing method, an image sensing program, arecording apparatus, a recording method, and a recording program thatallow desired cuts to be quickly searched when images are edited andallows a recording medium to be effectively used.

An embodiment of the present invention is an image sensing apparatusthat has an image sensor, an auto white balance adjustment section, achange point detection section, and an electronic mark data generationsection. The image sensor senses an image of an object and outputs animage signal corresponding to the image of the object. The auto whitebalance adjustment section automatically adjusts white balance of theimage signal corresponding to a change of the image signal. The changepoint detection section detects a change point of the white balancecorresponding to at least an adjustment value of the white balance whilethe image sensor is sensing the image of the object. The electronic markdata generation section generates electronic mark data for adjustedwhite balance based on time information of the change point detected bythe change point detection section.

An embodiment of the present invention is an image sensing method. Animage of an object is sensed. An image signal is output corresponding tothe image of the object. White balance of the image signal isautomatically adjusted corresponding to a change of the image signal. Achange point of the white balance is detected corresponding to at leastan adjustment value of the white balance while the image of the objectis being sensed. Electronic mark data for adjusted white balance isgenerated based on time information of the change point detected.

An embodiment of the present invention is an image sensing program thatcauses a microprocessor to execute an image sensing method. An image ofan object is sensed. An image signal is output corresponding to theimage of the object. White balance of the image signal is automaticallyadjusted corresponding to a change of the image signal. A change pointof the white balance is detected corresponding to at least an adjustmentvalue of the white balance while the image of the object is beingsensed. Electronic mark data for adjusted white balance are generatedbased on time information of the change point detected.

An embodiment of the present invention is a recording apparatus that hasan image sensor, an auto white balance adjustment section, a changepoint detection section, an electronic mark data generation section, anda recording section. The image sensor senses an image of an object andoutputs an image signal corresponding to the image of the object. Theauto white balance adjustment section automatically adjusts whitebalance of the image signal corresponding to a change of the imagesignal. The change point detection section detects a change point of thewhite balance corresponding to at least an adjustment value of the whitebalance while the image sensor is sensing the image of the object. Theelectronic mark data generation section generates electronic mark datafor adjusted white balance based on time information of the change pointdetected by the change point detection section. The recording sectionrecords the image signal whose white balance has been adjusted by thewhite balance adjustment section to a recording medium.

An embodiment of the present invention is a recording method. An imageof an object is sensed. An image signal is output corresponding to theimage of the object. White balance of the image signal is automaticallyadjusted corresponding to a change of the image signal. A change pointof the white balance is detected corresponding to at least an adjustmentvalue of the white balance while the image of the object is beingsensed. Electronic mark data for adjusted white balance is generatedbased on time information of the change point detected. The image signalwhose white balance has been adjusted is recorded to a recording medium.

An embodiment of the present invention is a recording program thatcauses a microprocessor to execute a recording method. An image of anobject is sensed. An image signal is output corresponding to the imageof the object. White balance of the image signal is automaticallyadjusted corresponding to a change of the image signal. A change pointof the white balance is detected corresponding to at least an adjustmentvalue of the white balance while the image of the object is beingsensed. Electronic mark data for adjusted white balance are generatedbased on time information of the change point detected. The image signalwhose white balance has been adjusted is recorded to a recording medium.

As described above, according to an embodiment of the present invention,the white balance of an image signal of an object that is shot isautomatically adjusted corresponding to a change of the image signal.While an object is being shot, a change point of the white balance isdetected corresponding to an adjustment value of the white balance. Withtime information for the detected change point, electronic mark data forthe adjusted white balance are generated. Thus, when an image signal isrecorded to a recording medium and the image signal is reproduced andedited, with the electronic mark data, scenes in which shootingenvironment and objects changed and the camera user changed his or hermind can be quickly searched.

In addition, according to an embodiment of the present invention, thewhite balance of an image signal of an object that is shot isautomatically adjusted corresponding to a change of the image signal.While an object is being shot, a change point of the white balance isdetected corresponding to an adjustment value of the white balance. Withtime information for the detected change point, electronic mark data forthe adjusted white balance are generated. The image signal whose whitebalance was adjusted is recorded to a recording medium. Thus, when theimage signal recorded on the recording medium is reproduced and edited,with the electronic mark data, scenes in which shooting environment andobjects changed and the camera user changed his or her mind can bequickly searched.

According to an embodiment of the present invention, user's operationsfor the video camera device and changes of operation information of thevideo camera device are affected by changes of objects and the change ofthe mind of the camera user. Thus, when these information and digitalimage signal are correlatively recorded, desired images can be quicklyand easily searched and reproduced.

In addition, since user's operations and changes of operationinformation of the video camera device are automatically recorded, theeffect of a quick and easy search for desired images can be accomplishedwithout needing to impose a special shooting operation on the user.

When an image signal is recorded, user's operations and changes ofoperation information of the video camera device are detected. Thedetected change and time information for the changes are correlativelyrecorded. Thus, the amount of record information is small.

In addition, according to an embodiment of the present invention, anauto tracking white balance operation is detected and recorded as achange of operation information of the video camera device. Thus, whenan image signal is reproduced, the user can easily know whether sceneswere shot indoor or outdoor. Thus, desired scenes can be quicklysearched.

In addition, when the operation of the auto tracking white balance isdetected, since changes of shutter speed, gain adjustment, and filterselection are considered, even if they offset a change of the whitebalance itself, changes of color temperature of an object that is shotcan be recorded.

These and other objects, features and advantages of the presentinvention will become more apparent in light of the following detaileddescription of a best mode embodiment thereof, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will become more fully understood from the followingdetailed description, taken in conjunction with the accompanyingdrawings, wherein similar reference numerals denote similar elements, inwhich:

FIG. 1 is a block diagram showing an example of the structure of a videocamera device according to an embodiment of the present invention;

FIG. 2 is a block diagram showing an example of the structure of an autowhite balance circuit;

FIG. 3 is a block diagram showing an example of the structure of arecording and reproducing portion;

FIG. 4 is a schematic diagram showing examples of reserved words thatdefine electronic mark data;

FIG. 5 is a schematic diagram showing an example of the data structureof electronic mark data;

FIG. 6 is a schematic diagram describing an example of a Take Meta DataFile;

FIG. 7 is a flow chart showing an example of a method of generating aTake Meta Data File;

FIG. 8 is a schematic diagram showing the data management structure ofan optical disc according to an embodiment of the present invention; and

FIG. 9 is a schematic diagram showing an example of a method ofrecording content data and a Take Meta Data File to an optical disc.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Next, an embodiment of the present invention will be described.According to the embodiment of the present invention, when a videocamera device shoots objects, it detects change points of its operationstates and correlatively records the change points and time informationthereof. When a digital image signal recorded by the video camera deviceis edited, with information of change points of operation states of thevideo camera device, points at which the brightness of objects andshooting environments have largely changed and points at which thecamera user changed his or her mind can be easily searched. Thus, theuser can effectively perform editing operations.

FIG. 1 shows an example of the structure of a video camera device 1according to an embodiment of the present invention. The video cameradevice 1 has a control device that generates predetermined controlsignals corresponding to a program. The control device has amicroprocessor such as a central processing unit (CPU), a random accessmemory (RAM), a read only memory (ROM), and a bus that connects thesedevices. The CPU generates various types of control signals and so forthcorresponding to programs and data that are pre-stored in the ROM. TheRAM is used as a work memory for the CPU. Each portion of the videocamera device 1 is controlled by the control device.

Light emitted from an object enters a charge coupled device (CCD) 11that is an image sensor. The image sensor is not limited to a CCD, butcan also be formed of any other image sensor such as a CMOS sensor, orsimilar device. The CCD 11 stores electric charges of individual pixelscorresponding to incident light. The CCD 11 converts the stored electriccharges into an electric signal and outputs the electric signal. At thispoint, a shutter 22 controls a storage time (shutter speed) of electriccharges to the CCD 11 to a constant time for each field. The storagetime of electric charges for each field is controlled with shuttercontrol information generated corresponding to a shutter speed that theuser selects on a shutter selection menu 23. The shutter controlinformation is supplied to both the shutter 22 and an auto white balancecircuit 14.

The video camera device 1 according to the embodiment of the presentinvention uses the so-called three-panel system that has three CCDs inthe CCD 11 corresponding to R (red), G (green), and B (blue). In thefollowing description, unless otherwise specified, an image signal iscomposed of three image signals of colors R, G, and B.

An image signal that is output from the CCD 11 is supplied to an A/Dconversion circuit 12. The A/D conversion circuit 12 converts the imagesignal into a digital image signal. The digital image signal is suppliedto a gain adjustment circuit 13. The gain adjustment circuit 13 sets again of the digital image signal corresponding to gain controlinformation. The gain adjustment circuit 13 supplies the digital imagesignal to an auto white balance circuit 14 and a luminance detection anddiaphragm control circuit 21. The gain control information is generatedcorresponding to an operation of a gain switch 24 that is one ofswitches of a user interface of the video camera device 1. The gaincontrol information is also supplied to the auto white balance circuit14.

The luminance detection and diaphragm control circuit 21 obtains theluminance of the digital image signal corresponding to a luminancecomponent thereof. The obtained luminance information is supplied to theauto white balance circuit 14. The luminance detection and diaphragmcontrol circuit 21 generates a diaphragm control signal with which thediaphragm of an optical system 10 is controlled with the obtainedluminance information. The diaphragm control information is supplied toa lens diaphragm mechanism 20. The lens diaphragm mechanism 20 adjuststhe diaphragm of the optical system 10 corresponding to the supplieddiaphragm control information so that the amount of incident light ofthe CCD 11 becomes a predetermined amount.

Disposed on the light incident side of the optical system 10 is anoptical filter mechanism that has a plurality of optical filters. Theoptical filter mechanism includes a plurality of optical filters thatgradually decrease the amount of incident light of the optical system 10and a plurality of optical filters that compensate color temperature ofincident light. The optical filter mechanism selects one from theplurality of optical filters corresponding to a user's operation in afilter selection mechanism 25. Filter selection information that isoutput from the filter selection mechanism 25 is supplied as filterinformation to the auto white balance circuit 14.

The auto white balance circuit 14 adjusts the white balance of the inputdigital image signals of R, G, and B. The auto white balance circuit 14outputs digital image signals of R, G, and B that have been whitebalanced. The auto white balance circuit 14 performs a so-called autotracking white balance operation that causes the input digital imagesignals to be automatically white balanced corresponding to the changesthereof.

Next, the white balance will be described in brief. First of all, colortemperature will be described. Color temperature is a color of lightemitted from a perfect black body that is heated. The color of theperfect black body and temperature thereof are correlatively definedwith a relationship of 1 to 1. The temperature necessary to radiate theperfect black body with light of a color is the unit of colortemperature. The color temperature is represented by “K” (Kelvin), whichis the unit of thermodynamic temperature. The color of light emittedfrom the perfect black body varies from red to white to blue as thetemperature rises. Thus, a totally reddish screen can be called “lowcolor temperature,” whereas a totally bluish screen can be called “highcolor temperature.” The color temperature of a 60 W tungstenincandescent lamp as a regular light source is around 2800 K. The colortemperature of noon sunlight is around 5600 K. The color temperature ofa daylight color fluorescent lamp is around 6500 K.

White balance, which is one of adjustment functions of a video cameraand so forth, is a function that adjusts the balance of colors of red(R), green (G), and blue (B) so that a white object is shot as anunbiased white image under a light source of particular colortemperature.

When the white balance of the video camera is adjusted under a lightsource of low color temperature, for example a 60 W tungstenincandescent lamp, a white object is shot as an white image. However, ifthe same object is shot with a light source of higher color temperaturesuch as noon sunlight with the same white balance setting, the whiteobject is shot as a bluish image. Thus, when an object is shot with alight source of different color temperature, it is necessary to adjustthe white balance.

FIG. 2 shows an example of the structure of the auto white balancecircuit 14. In the following description, a red (R) digital image signalis referred to as an R signal. This also applies to green (G) and blue(B) digital image signals. The R signal of the R signal, G signal, and Bsignal that are input to the auto white balance circuit 14 is suppliedto an integration circuit 52 through a variable gain amplifier 50. The Gsignal is supplied to an integration circuit 53. The B signal issupplied to an integration circuit 54 through a variable gain circuit51.

The integration circuits 52, 53, and 54 integrate R, G, and B signalsfor one screen each and output the resultant signals. Outputs of theintegration circuits 52, 53, and 54 are supplied to a comparison circuit55. The comparison circuit 55 compares the levels of the R, G, and Bsignals integrated for one screen each. The gains of the amplifiers 50and 51 are adjusted so that the levels of the R signal, G signal, and Bsignal become identical. In other words, by adjusting the gains of theamplifier 50 and the amplifier 51, the levels of the R signal and Bsignal are relatively varied compared to the level of the G signal sothat the levels of the R, G, and B signals become identical. In thismanner, the white balance is adjusted. The auto white balance circuit 14performs this process for each field or at predetermined intervals so asto follow up the white balance adjustment operation for changes of theR, G, and B signals.

A digital image signal that is output from the auto white balancecircuit 14 is obtained from input sides of the integration circuits 52,53, and 54. An adjustment value R and an adjustment value B are outputwith which the gains of the variable gain amplifiers 50 and 51 areadjusted corresponding to the compared result of the comparison circuit55.

In addition, the auto white balance circuit 14 obtains brightnessinformation of the digital image signal with the luminance informationand diaphragm control information supplied from the luminance detectionand diaphragm control circuit 21, the shutter control informationsupplied from the shutter selection menu 23, the gain controlinformation supplied from a gain switch 24, and the filter informationsupplied from the filter selection mechanism 25. The auto white balancecircuit 14 controls the follow-up speed of the white balance adjustmentagainst the changes of the digital image signals corresponding to thebrightness information.

In other words, if the brightness of an object has sharply varied, forexample if the shooting location has been changed from indoor to outdooror from outdoor to indoor, the follow-up speed of the auto trackingwhite balance is increased so as to deal with the sharp change of thecolor temperature of the object. When it has been determined that theauto tracking white balance operation has caused the white balanceadjustment to converge, the follow-up speed of the auto tracking whitebalance operation is decreased.

For example, if the shooting location is changed from outdoor to indoorin the daytime, the indoor is much darker than the outdoor in thedaytime. The light amount of the indoor is much smaller than the lightamount of the outdoor. Thus, when the shooting location is changed fromindoor to outdoor, luminance information may largely vary. If theluminance information largely varies, the luminance detection anddiaphragm control circuit 21 generates diaphragm control informationthat causes the diaphragm to decrease the change of the luminanceinformation. In this case, depending on the shooting conditions and soforth, the user may increase the shutter speed, decrease the gain of thegain adjustment circuit 13 with the gain switch 24, and select a highattenuation filter with the filter selection mechanism 25. Thus, whenthe brightness of the object largely varies, the luminance information,diaphragm control information, shutter control information, gain controlinformation, and/or filter information also vary.

Occasionally, when moving the shooting location from indoor to outdoor,the camera user may for example adjust the shutter speed, adjust thegain, and select a filter. In this case, since the luminance signal doesnot largely vary, the diaphragm control information does not largelyvary. However, since the shutter control information, gain controlinformation, and/or filter information vary, it can be used to determinewhether the shooting location is indoor or outdoor.

The changes of the diaphragm control information, shutter controlinformation, gain control information, and filter information can becorrelated with the change of the brightness of the object. These itemsof information can be used to indicate that the brightness of the objecthas varied.

For example, the luminance information, diaphragm control information,shutter control information, gain control information, and filterinformation are supplied to the comparison circuit 55. The comparisoncircuit 55 determines whether supplied values exceed their thresholdvalues. When one of the values exceeds the corresponding thresholdvalue, it is determined that the brightness of the object has largelyvaried and the shooting location has been changed from indoor to outdoor(or from outdoor to indoor). Alternatively, when the values of aplurality of items of information exceed their threshold values, it maybe determined that the shooting location has been changed from indoor tooutdoor (or from outdoor to indoor).

When it has been determined that the shooting location had been changedfrom indoor to outdoor (or from outdoor to indoor), the adjustment valueR and the adjustment value B supplied to the amplifiers 50 and 51 arecontrolled so that the time constant of a feedback loop composed of theamplifier 50 and/or 51 and the comparison circuit 55 decreases and thefollow-up speed of the auto tracking white balance increases. Thecomparison circuit 55 monitors outputs of the integration circuits 52,53, and 54. When the changes of the outputs become equal to or smallerthan their threshold values, it is determined that the white balanceadjustment has been converged. At this point, the adjustment values Rand B are controlled so that the time constant of the feedback loopincreases and the follow-up speed of the auto tracking white balanceoperation decreases.

Returning to FIG. 1, the R, G, and B digital image signals that areoutput from the auto white balance circuit 14 are sent to a recordingand reproducing portion 2. The recording and reproducing portion 2 maybe integrated with the video camera device 1. Alternatively, therecording and reproducing portion 2 may be independent from the videocamera device 1 connected thereto with a cable.

FIG. 3 shows an example of the structure of the recording andreproducing portion 2. A spindle motor 112 drives the rotations of anoptical disc 100 at constant linear velocity (CLV) or constant angularvelocity (CAV) corresponding to a spindle motor drive signal receivedfrom a servo control portion 115.

A pickup portion 113 controls an output of laser light corresponding toa record signal supplied from a signal process portion 116 and recordsthe record signal to the optical disc 100. The pickup portion 113focuses laser light on the optical disc 100. In addition, the pickupportion 113 converts light reflected from the optical disc 100 intoelectricity, generates a current signal, and supplies the current signalto a radio frequency (RF) amplifier 114. A predetermined position isradiated with the laser light corresponding to a servo signal suppliedfrom the servo control portion 115 to the pickup portion 113.

The RF amplifier 114 generates a focus error signal, a tracking errorsignal, and a reproduction signal corresponding to the current signalsupplied from the pickup portion 113. The tracking error signal and thefocus error signal are supplied to the servo control portion 115. Thereproduction signal is supplied to the signal process portion 116.

The servo control portion 115 controls the focus servo operation and thetracking servo operation. In reality, the servo control portion 115generates a focus servo signal and a tracking servo signal correspondingto the focus error signal and the tracking error signal supplied fromthe RF amplifier 114 and supplies the generated signals to respectiveactuators (not shown) of the pickup portion 113. The servo controlportion 115 generates a spindle motor drive signal with which thespindle motor 112 is driven and controls the spindle servo operationthat rotates the optical disc 100 at a predetermined rotation speed.

In addition, the servo control portion 115 performs a thread controlthat radially moves the video camera device 1 and changes the radiationposition of laser light. The signal read position of the optical disc100 is set by a control portion 120. The control portion 120 controlsthe position of the pickup portion 113 to read the signal from thepreset read position.

The signal process portion 116 modulates record data that are input froma memory controller 117, generates a record signal, and supplies therecord signal to the pickup portion 113. The signal process portion 116demodulates a reproduction signal supplied from the RF amplifier 114,generates reproduction data, and supplies the reproduction data to thememory controller 117.

The memory controller 117 stores record data supplied from a dataconversion portion 119 to a memory 118, reads the record data therefrom,and supplies the record data to the signal process portion 116. Inaddition, the memory controller 117 stores reproduction data suppliedfrom the signal process portion 116 to the memory 118, reads thereproduction data therefrom, and supplies the reproduction data to thedata conversion portion 119.

A digital image signal that is output from the video camera device 1 issupplied to the data conversion portion 119. The video camera device 1collects sound with a microphone (not shown), converts the sound into adigital audio signal, and outputs the digital audio signal. The digitalaudio signal is sent from the video camera device 1 to the recording andreproducing portion 2. The digital audio signal is supplied to the dataconversion portion 119.

The data conversion portion 119 compression-encodes the supplied digitalimage signal in a mode designated by the control portion 120corresponding to a compression encoding system such as the MPEG2 systemand generates a record digital image signal. In addition, the dataconversion portion 119 compression-encodes the supplied digital audiosignal in a system designated by the control portion 120 and outputs theencoded signal as a record digital audio signal. The digital audiosignal may not be compression-encoded, but may be directly output as alinear pulse code modulation (PCM) digital audio signal.

The record digital image signal and the record digital audio signal thatare processed by the data conversion portion 119 are supplied to thememory controller 117.

When necessary, the data conversion portion 119 decodes the reproductiondigital image signal and reproduction digital audio signal supplied fromthe memory controller 117, converts them into predetermined formatsignals, and outputs the converted signals.

The control portion 120 is composed of a CPU, memories such as a ROM anda RAM, and a bus that connects these devices. The control portion 120controls the entire recording and reproducing portion 2. When therecording and reproducing portion 2 is integrated with the foregoingvideo camera device 1, a control portion (not shown) of the video cameradevice 1 and the control portion 120 may be used in common. The ROMpre-stores an initial program that is read when the CPU gets started anda program and so forth with which the recording and reproducing portion2 is controlled. The RAM is used as a work memory of the CPU. Thecontrol portion 120 controls the video camera portion.

In addition, the control portion 120 provides a file system that recordsdata to the optical disc 100 corresponding to a program that ispre-stored in the ROM and reproduces the record data from the opticaldisc 100. In other words, the recording and reproducing portion 2records data to the optical disc 100 and reproduces data therefrom underthe control of the control portion 120.

An operation portion 121 is operated by the user. The operation portion121 supplies an operation signal corresponding to a user's operation tothe control portion 120. The control portion 120 controls the servocontrol portion 115, the signal process portion 116, the memorycontroller 117, and the data conversion portion 119 corresponding to theoperation signal and so forth supplied from the operation portion 121 tocause these devices to execute the recording and reproducing processes.

In addition, the control portion 120 sets bit rate, frame rate, picturesize, and picture aspect ratio of, for example, the record digital imagesignal corresponding to the operation signal supplied from the operationportion 121. In addition, the ON/OFF of the compression encoding processfor the record audio data and setting of bit resolution may be performedby the operation portion 121. Control signals corresponding to thesesettings are supplied to the memory controller 117 and the dataconversion portion 119.

Next, generation and recording of electronic mark data according to anembodiment of the present invention will be described. The video cameradevice 1 according to the embodiment of the present invention detectsvarious operations that the video camera device 1 performs to shoot anobject and record camera operation information that represents thevarious operations as electronic mark data to a recording medium.

Referring to FIG. 1, a focus change detection circuit 27 determineswhether the focus adjustment value of a focus mechanism 26 exceeds apredetermined threshold value and detects a focus change point. Forexample, the focus change detection circuit 27 differentiates the focusadjustment value and determines whether the differentiated value exceedsa predetermined threshold value. When the focus adjustment value exceedsthe predetermined threshold value, the focus change detection circuit 27supplies a pulse as a detection result to an electronic mark datageneration circuit 40. The electronic mark data generation circuit 40generates an electronic mark that represents the supplied detectionresult, correlates the electronic mark with time information, anddescribes the correlated data to an electronic mark list 42. Theelectronic mark list 42 is stored in a RAM as a work memory of a CPU(not shown).

According to this embodiment of the present invention, time informationis generated corresponding to frame numbers generated by a frame numbergeneration circuit 41. In other words, the frame number generationcircuit 41 generates frame numbers of a digital image signal of an imageof an object which the video camera device 1 is shooting. When the framenumbers are used as time information corresponding to electronic markdata, the relationship between electronic mark data and a digital imagesignal can be easily obtained. Time information is not limited to framenumbers, but information that represents real time can also be used.Alternatively, time codes defined in the Society of Motion Picture andTelevision Engineers (SMPTE) may be used. Alternatively, original timeinformation may be used.

This mechanism can be applied to other types of change points. A zoomchange detection circuit 29 determines whether the zoom adjustment valueof a zoom mechanism 28 exceeds a predetermined threshold value. When thezoom adjustment value exceeds the predetermined threshold value, thezoom change detection circuit 29 detects a zoom change point. Thedetection result of the zoom change detection circuit 29 is supplied tothe electronic mark data generation circuit 40. The electronic mark datageneration circuit 40 correlates the detection result with timeinformation and describes the correlated data to the electronic marklist 42. A diaphragm change detection circuit 30 determines whether thediaphragm adjustment value of the lens diaphragm mechanism 20 exceeds apredetermined threshold value. When the diaphragm adjustment valueexceeds the predetermined value, the diaphragm change detection circuit30 detects a diaphragm change value. The detection result of thediaphragm change detection circuit 30 is supplied to the electronic markdata generation circuit 40. The electronic mark data generation circuit40 correlates the detection result with time information and describesthe correlated data to the electronic mark list 42.

Change detection circuits 15R and 15L determine whether the adjustmentvalues R and B of white balance adjustment results of the auto whitebalance circuit 14 exceed their threshold values and detect changepoints of the adjustment values R and L. The detection results of thechange points of the adjustment values R and B are supplied to a whitebalance change detection circuit 16.

On the other hand, as described above, the auto white balance circuit 14checks the change of the brightness of the object with the luminanceinformation, diaphragm control information, shutter control information,gain control information, and filter control information and detectswhether the shooting location has been changed from indoor to outdoor(or from outdoor to indoor). The auto white balance circuit 14 controlsthe follow-up speed for the white balance adjustment against the digitalimage signal corresponding to the detection result. Indoor/outdoorchange information of the detection result is supplied to the whitebalance change detection circuit 16.

When the white balance change detection circuit 16 has detected anychange with the change detection result of the change detection circuit15R, the change point detection result of the change detection circuit15B, and the indoor/outdoor change information supplied from the autowhite balance circuit 14, the white balance change detection circuit 16determines that the white balance has changed and supplies the detectionresults to the electronic mark data generation circuit 40.

In other words, when the shutter speed, the gain of the gain adjustmentcircuit 13, or the filter has been changed, as described above, it canbe considered that the shooting location has been changed from indoor tooutdoor (or from outdoor to indoor). It can be thought that the colortemperature has changed under these conditions. At this point, if theshutter speed, gain, or filter is changed corresponding to the change ofthe color temperature, the change of the color temperature of the objectis offset with the change of the shutter speed, gain, or filter. Thus,there is a possibility the auto white balance circuit 14 cannot detectthe white balance corresponding to the R signal, the G signal, and the Bsignal. Thus, according to the embodiment, when it has been detectedthat the shutter speed, the gain, or the filter has changed, it isconsidered that the shooting location has been changed from indoor tooutdoor (or from outdoor to indoor). Thus, it is considered that thechange of the white balance has been detected.

The electronic mark data generation circuit 40 correlates the suppliedwhite balance change detection results with time information anddescribes the correlated data to the electronic mark list 42.

Detection circuits (not shown) determine whether the shutter controlinformation, the gain control information, and the filter informationexceed predetermined threshold values and detect change points. Thedetection results are supplied to the electronic mark data generationcircuit 40. The electronic mark data generation circuit 40 correlatesthe detection results with time information and describes the correlateddata to the electronic mark list 42.

When a clip of a digital image signal of an image of an object that wasshot is recorded to the optical disc 100 by the recording andreproducing portion 2, the electronic mark list 42 is stored in one TakeMeta Data File corresponding to the clip and recorded to the opticaldisc 100. In this example, a clip is a group of data after a shootingoperation is started until it is stopped. For example, one clip iscomposed of a digital image signal (and a digital audio signal) that areinput after a start button (not shown) is pressed until a stop button(not shown) is pressed (or the start button is released).

A Take Meta Data File is a file that contains at least one unit ofelectronic mark data that are generated when one clip is shot. Meta dataare more highly ranked data than regular data and function as an indexthat represents the contents of various types of data. A Take Meta DataFile is meta data that represent an index of content data (digital imagesignal and digital audio signal) for each clip.

A digital image signal (and a digital audio signal) that compose a clipare supplied from the video camera device 1 to the recording andreproducing portion 2 and supplied to the data conversion portion 119.In addition, the Take Meta Data File is output from the video cameradevice 1 and supplied to, for example, the control portion 120 of therecording and reproducing portion 2. The control portion 120 suppliesthe Take Meta Data File to the memory controller 117. At proper timings,the memory controller 117 stores the Take Meta Data File to the memory118, reads it therefrom, and supplies it as record data to the signalprocess portion 116.

A Take Meta Data File and a clip may be recorded to the same recordingmedium by the recording and reproducing portion 2. Alternatively, a TakeMeta Data File and a clip may be recorded or stored to differentrecording mediums. In other words, a clip may be recoded to the opticaldisc 100 by the recording and reproducing portion 2, whereas a Take MetaData File may be stored in a detachable non-volatile semiconductormemory such as a flash memory attached to the video camera device 1.

Next, take meta data will be described in detail. FIG. 4 shows examplesof reserved words that define electronic mark data. Electronic mark datadefined with reserved words shown in FIG. 4 can be described in a TakeMeta Data File. Each item name of electronic mark data is composed of anunderscore “_” and a character string that represents the content of theitem, for example, “Flash,” “WhiteBalanceChange,” or the like. Besidesthose shown in FIG. 4, other electronic mark data can be additionallydefined. Rules of reserved words that define electronic mark data arenot limited to those examples.

“_RecStart” is a shooting start mark that represents a record startposition. “_RecEnd” is a shooting end mark that represents a record endposition. “_ShotMark1” and “_ShotMark2” are shot marks that representany position such as a noticeable time point. “_Cut” is a cut mark thatrepresents a cut position. “_Flash” is a flash mark that represents aflash detection position.

“_FilterChange” is a filter change mark that represents the position atwhich a lens filter of the image sensing apparatus was changed.“_FilterChange” corresponds to a detected result of a change point ofthe foregoing filter information. “_ShutterSpeedChange” is a shutterspeed change mark that represents the position at which the shutterspeed of the image sensing apparatus was changed. “_ShutterSpeedChange”corresponds to a detection result of a change point of theabove-described shutter control information. “_GainChange” is a gainchange mark that represents the position at which the gain was changed.“_GainChange” corresponds to a detection result of a change point of theabove-described filter control information. “_WhiteBalanceChange” is awhite balance change mark that represents a position at which whitebalance was changed. “_WhiteBalanceChange” corresponds to a detectionresult of a change point of the above-described white balance.

“_FilterChange,” “_ShutterSpeedChange,” and “_GainChange” may bedescribed independently or converted as indoor/outdoor changeinformation into “_WhiteBalanceChange.” Alternatively, these independentmarks and their converted marks may be described in combination.

“_OverBrightness” is a mark that represents the position at which theoutput level of an image signal exceeds a limit value.“_OverAudioLimiter” is a large sound volume mark that represents theposition at which the output level of an audio signal exceeds a limitvalue. In marks from “_FilterChange” to “_OverAudioLimiter,” a changethat takes place while a digital image signal is being recorded isconsidered.

“_In-XXX” is an edit start mark that represents the cut start positionof a cut or a material. “_Out-XXX” is an edit end mark that representsthe cut end position of a cut or a material. Whenever the edit startpoint (in-point) and the edit end pint (out-point) are added, a numberor an alphabetic character is sequentially numbered in the portion “XXX”such as “_In-001,” “_In-002,” and so forth. “_KeyFrame” is a key framemark that represents the position of a representative frame of amaterial.

Since the above-defined electronic mark data are used as indexinformation when video data are roughly edited, desired video scenes canbe effectively selected.

FIG. 5 shows an example of the data structure of electronic mark data.As described in FIG. 4, electronic mark data are meta data in whichfeatures of video scenes are represented with text data and correlatedwith a digital image signal. Electronic mark data are encoded with thekey length value (KLV) system and can be recorded and transmitted in forexample the serial data interface (SPI). FIG. 5 shows the format ofelectronic mark data that have been KLV encoded. This format is based onthe SMPTE 335M/RP210A meta data dictionary.

Electronic mark data that have been KLV encoded have a “key” portion of16 bytes long, an “L (length)” portion of one byte long, and a “Value”portion of a maximum of 32 bytes long. The “Key” portion is anidentifier that represents a data item that has been KLV encoded on thebasis of the SMPTE 335M/RP210A. In this example, the “Key” portion is avalue that represents electronic mark data. The “L” portion representsthe data length after the “L”portion in bytes. The “L” portion canrepresent the data length of a maximum of 32 bytes. The “Value” portionis an area of text data for electronic mark data.

FIG. 6 shows an example of a description of a Take Meta Data File.According to this embodiment, a Take Meta Data File is described withthe extensible markup language (XML). The XML is a language thatdescribes information with tags that can be originally defined. With theXML, tags allow described information to be specially defined. Since theXML allows each tag of an XML file to contain parameters that havemeaning, new parameters can be easily defined. Thus, the XML has highexpansibility.

Next, tags of XML files will be described in brief. Generally, tags arecomposed of a pair of symbols that represent the beginning and end of arange. Tags can be embedded in a text. For example, a tag thatrepresents the beginning of a range is described by surrounding apre-defined character string with a pair of symbols “<” and “>” (thispair is called a beginning tag). A tag that represents the end of therange is described by surrounding the same character string with a pairof symbols “</” and “>” (this pair is called an end tag). In a rangesurrounded by a pair of tags, any meaning can be defined with acharacter string surrounded by a pair of symbols “<” and “>” (or a pairof symbols “</” and “>”). In addition, predetermined parameters can bedescribed in tags. Tags can be nested. In an XML file, nest levels arerepresented with the depths of indents.

As shown in FIG. 6, a Take Meta Data File contains at least a videoinformation portion 301, a video file name portion 302, and a markportion 303. The video information portion 301 is video information ofthe video camera device 1. The video file name portion 302 is a filename of a content data file such as a digital image signal and a digitalaudio signal. The mark portion 303 is an electronic mark data.Definition names in tags are not limited to those shown in FIG. 6.Alternatively, other definition names may be used.

The video information portion 301 is defined between a tag “<Device>”and a “tag </Device>.” For example, a tag “<Model name=“ABC-0123” serialNo=“12-34-56-78”>” defines a model name of the video camera device 1.The video file name portion 302 defined between a tag“<ComponentMaterial>” and a tag “</ComponentMaterial>.” For example, afile name “video1.mxf” of a digital image signal file of a content datafile is defined as a tag “<Video src=“video1.mxf”>. A file name“audio1.mxf” of a digital audio signal file is defined as a tag<Audio.src=“audio1.mxf”>where “.mxf” represents that the format of thesefiles is based on “material exchange format.”

The mark portion 303 is defined between a tag “<EssenceMarkTable>” and atag “</EssenceMarkTable>” as electronic mark data and time informationcorresponding thereto. In a tag “<EssenceMark value =“_XXXXXX”>”, aportion that is surrounded by double quotation marks ””describes an itemname of electronic mark data. As shown in FIG. 4, an underscore “_” isimmediately followed by a character string that clearly represents thecontent (for example “Flash,” “WhiteBalanceChange,” and so forth). Timeinformation corresponding to electronic mark data is defined between atag “<EssenceMark value=”“_XXXXXX”> and a tag “/EssenceMark.”

In FIG. 6, a frame number of a digital image signal is used as timeinformation. However, time information may be real time information thatincludes date and time information.

FIG. 7 is a flow chart showing an example of a generation method of aTake Meta Data File. When the video camera device 1 starts shooting aclip (at step S10), it is determined whether an event for electronicmark data has been detected (at step S11). When the determined resultrepresents that an event for electronic mark data has been detected, theflow advances to step S12. At step S12, electronic mark datacorresponding to the event is generated. At step S13, the electronicmark data generated at step S12 is stored in a memory or the like.

When the shutter speed is changed on the shutter selection menu 23,shutter control information is supplied to the electronic mark datageneration circuit 40. The electronic mark data generation circuit 40generates corresponding electronic mark data. The electronic mark datacomposed of a reserved word for a change of the shutter speed (forexample “_ShutterSpeedChange” in FIG. 4) and time information of changedetection time that are correlated and stored in a memory or the like.In addition, the shutter control information is also supplied to theauto white balance circuit 14. The auto white balance circuit 14generates indoor/outdoor change information corresponding to a change ofthe shutter speed. The indoor/outdoor change information is supplied tothe white balance change detection circuit 16. The white balance changedetection circuit 16 generates electronic mark data that represent achange of the white balance. The electronic mark data are composed of areserved word for a change of the white balance (for example“_WhiteBalanceChange ” shown in FIG. 4) and time information of changedetection time that are correlated and stored in the memory or the like.

At step S14, content data of a digital image signal and a digital audiosignal are recorded to the disc 100. At step S15, it is determinedwhether the content data have been recorded to the disc 100. When thecontent data have not been recorded, the flow returns to step S11. Inthe real system, the electronic mark data generation process at stepsS11 to S13 and the content data record process at step S14 are performedin parallel. Thus, while content data are being recorded, electronicmark data are generated and stored.

When it has been determined that a clip has been recorded, the flowadvances to step S16. With electronic mark data stored in the memory inthe process from step S11 to step S13, a Take Meta Data File is created.At step S17, the Take Meta Data File is recorded to the recordingmedium.

FIG. 8 schematically shows a data management structure of the opticaldisc 100 according to an embodiment of the present invention. As shownin FIG. 8, on the optical disc 100, data are hierarchically managed witha directory structure. Under a directory “ROOT,” directories for clipsare created. Under the directory for clips, a file of a clip, namely atleast a content data file of the clip and a Take Meta Data File of theclip, are recorded.

In the example shown in FIG. 8, under the directory “ROOT,” directories“CLIP a,” “CLIP b,” and so forth for these clips are created. Under thedirectory “CLIP a,” as a content data file of the clip, a digital imagesignal file “VIDEO a” and a Take Meta Data File “TAKE META DATA a”corresponding to the content data file are recorded. This relationshipapplies to the directories “CLIP b” and so forth.

FIG. 9 shows an example of the record method of content data and a TakeMeta Data File to the optical disc 100. The optical disc 100 has aplurality of tracks as a record area. In FIG. 9, the plurality of tracksare concentrically illustrated. However, actually, the plurality oftracks are helically formed. Data are successively recorded to helicaltracks from the center to the outer periphery of the optical disc 100.

At step S10 shown in FIG. 7, the recording of the clip is started. Atstep S14, content data are written from a position A on the optical disc100. While data are present, the record point moves on helical tracks tothe outer periphery side. When the clip has been recorded at step S15,the record point has moved to a position B on the optical disc. Afterthe clip has been recorded, a Take Meta Data File is recorded to theoptical disc 100 (at step S17). In the example shown in FIG. 9, the TakeMeta Data File is recorded after a position C that is a later recordposition than the record end position B of the clip until a position Don the optical disc.

In the foregoing description, it was assumed that a Take Meta Data Fileis recorded on the outer periphery side of content data. However, a TakeMeta Data File may be recorded on the inner periphery side of contentdata. In the foregoing description, after one content is recorded, takemeta data corresponding thereto is recorded. After a plurality of filesof content data are recorded, a plurality of Take Meta Data Filescorresponding thereto may be recorded.

In the foregoing description, the recording medium on which a digitalimage signal is recorded is a recordable optical disc. However, therecording medium may be a magnetic tape, a semiconductor memory, etc.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alternations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

Obviously, numerous modifications and variations of the presentinvention are possible in light of the above teachings. It is thereforeto be understood that within the scope of the appended claims, thepresent invention may be practiced otherwise than as specificallydescribed herein.

1. An image sensing apparatus, comprising: an image sensor configured tosense an image of an object and to output an image signal correspondingto the image of the object; an auto white balance adjustment circuitconfigured to automatically adjust white balance of the image signalbased on a change of the image signal; a change point detection circuitconfigured to detect a change point of the white balance based on atleast an adjustment value of the white balance while said image sensorsenses the image of the object; and an electronic mark data generationcircuit configured to generate electronic mark data for adjusted whitebalance based on time information of the change point detected by saidchange point detection circuit.
 2. The image sensing apparatus as setforth in claim 1, wherein said change point detection circuit detects achange point of the white balance also based on information thatrepresents that brightness of the object changed while said image sensorsenses the image of the object.
 3. The image sensing apparatus as setforth in claim 2, further comprising: a shutter speed control circuitconfigured to control a shutter speed of said image sensing unit,wherein the information that represents that the brightness of theobject changed is shutter control information of said shutter controlcircuit.
 4. The image sensing apparatus as set forth in claim 2, furthercomprising: a gain control circuit configured to control the gain of theimage signal, wherein the information that represents that thebrightness of the object changed is a gain control signal of said gaincontrol circuit.
 5. The image sensing apparatus as set forth in claim 2,further comprising: a filter selection circuit configured to select atleast one of optical filters that restrain or compensate incident lightthat enters said image sensor, wherein information that represents thatthe brightness of the object changed is filter selection information ofsaid filter selection circuit.
 6. The image sensing apparatus as setforth in claim 2, wherein the change point of the brightness of theobject is information based on a luminance component of the imagesignal.
 7. The image sensing apparatus as set forth in claim 2, furthercomprising: a diaphragm configured to restrain an optical path of theincident light that enters said image sensor.
 8. The image sensingapparatus as set forth in claim 2, wherein said auto white balanceadjustment circuit controls a follow-up speed that follows the change ofthe image signal corresponding to the information that represents thatthe brightness changed.
 9. The image sensing apparatus as set forth inclaim 1, wherein the time information is a frame number of the imagesignal.
 10. The image sensing apparatus as set forth in claim 1, furthercomprising: a shutter speed changing circuit configured to change ashutter speed of said image sensor, wherein said electronic mark datageneration circuit is further configured to generate electronic markdata for the shutter speed based on time information of a change pointof the shutter speed.
 11. The image sensing apparatus as set forth inclaim 1, further comprising: a gain changing circuit configured tochange a gain of the image signal, and wherein said electronic mark datageneration circuit is further configured to generate electronic markdata for the gain corresponding to time information of a change point ofthe gain.
 12. The image sensing apparatus as set forth in claim 1,further comprising: a filter selection circuit configured to select atleast one of optical filters that restrain or compensate incident lightthat enters said image sensor, wherein said electronic mark datageneration circuit is further configured to generate electronic markdata for the optical filter corresponding to time information of aselection point of the optical filter selected by said filter selectioncircuit.
 13. The image sensing apparatus as set forth in claim 1,wherein said electronic mark data generation circuit is furtherconfigured to package electronic mark data generated after said imagesensor issues a sensing start command until said image sensor unitissues a sensing end command.
 14. The image sensing apparatus as setforth in claim 1, further comprising: a recording circuit configured torecord the image signal to a recording medium, wherein the electronicmark data generated by said electronic mark data generation circuit isrecorded to the recording medium along with the image signal.
 15. Theimage sensing apparatus as set forth in claim 1, further comprising: arecording circuit configured to record the image signal to a recordingmedium, wherein the electronic mark data generated by said electronicmark data generation circuit is recorded or stored to a recording mediumor a storing medium different from the recording medium for the imagesignal.
 16. An image sensing method, comprising: sensing an image of anobject and outputting an image signal corresponding to the image of theobject; automatically adjusting white balance of the image signalcorresponding to a change of the image signal; detecting a change pointof the white balance corresponding to at least an adjustment value ofthe white balance while the image of the object is being sensed at theimage sensing; and generating electronic mark data for adjusted whitebalance based on time information of the change point detected at thedetecting a change point.
 17. An image sensing program that causes amicroprocessor to execute an image sensing method, comprising: sensingan image of an object and outputting an image signal corresponding tothe image of the object; automatically adjusting white balance of theimage signal corresponding to a change of the image signal; detecting achange point of the white balance corresponding to at least anadjustment value of the white balance while the image of the object isbeing sensed at the image sensing; and generating electronic mark datafor adjusted white balance based on time information of the change pointdetected at the detecting a change point.
 18. A recording method,comprising: sensing an image of an object and outputting an image signalcorresponding to the image of the object; automatically adjusting whitebalance of the image signal corresponding to a change of the imagesignal; detecting a change point of the white balance corresponding toat least an adjustment value of the white balance while the image of theobject is being sensed at the image sensing; generating electronic markdata for adjusted white balance based on time information of the changepoint detected at the detecting a change point; and recording the imagesignal whose white balance has been adjusted at the automaticallyadjusting white balance to a recording medium.
 19. A recording programthat causes a microprocessor to execute a recording method, comprising:sensing an image of an object and outputting an image signalcorresponding to the image of the object; automatically adjusting whitebalance of the image signal corresponding to a change of the imagesignal; detecting a change point of the white balance corresponding toat least an adjustment value of the white balance while the image of theobject is being sensed at the image sensing; generating electronic markdata for adjusted white balance based on time information of the changepoint detected at the detecting a change point; and recording the imagesignal whose white balance has been adjusted at the automaticallyadjusting white balance to a recording medium.
 20. An image sensingapparatus, comprising: means for sensing an image of an object and tooutput an image signal corresponding to the image of the object; meansfor automatically adjusting white balance of the image signal based on achange of the image signal; means for detecting a change point of thewhite balance based on at least an adjustment value of the white balancewhile said means for sensing is sensing the image of the object; andmeans for generating electronic mark data for adjusted white balancebased on time information of the change point detected by said means fordetecting a change point.
 21. The image sensing apparatus as set forthin claim 20, wherein said means for detecting a change point detects achange of white balance also corresponding to information thatrepresents that brightness of the object changed while said means forsensing is sensing the image of the object.
 22. The image sensingapparatus as set forth in claim 21, further comprising: means forcontrolling a shutter speed of said means for sensing, wherein theinformation that represents that the brightness of the object changed isshutter control information of said means for controlling shutter speed.23. The image sensing apparatus as set forth in claim 21, furthercomprising: means for controlling gain of the image signal, wherein theinformation that represents that the brightness of the object changed isa gain control signal of said means for controlling gain.
 24. The imagesensing apparatus as set forth in claim 21, further comprising: meansfor selecting at least one of optical filters that restrain orcompensate incident light that enters said means for sensing, whereininformation that represents that the brightness of the object changed isfilter selection information of said means for selecting.
 25. The imagesensing apparatus as set forth in claim 21, wherein the change point ofthe brightness of the object is information based on a luminancecomponent of the image signal.
 26. The image sensing apparatus as setforth in claim 21, further comprising: means for restraining an opticalpath of the incident light that enters said means for sensing.
 27. Theimage sensing apparatus as set forth in claim 21, wherein said means forautomatically adjusting white balance controls a follow-up speed thatfollows the change of the image signal based on the information thatrepresents that the brightness changed.
 28. The image sensing apparatusas set forth in claim 20, wherein the time information is a frame numberof the image signal.
 29. The image sensing apparatus as set forth inclaim 20, further comprising: means for changing a shutter speed of saidmeans for sensing, wherein said means for generating electronic markdata further generates electronic mark data for the shutter speed basedon time information of a change point of the shutter speed.
 30. Theimage sensing apparatus as set forth in claim 20, further comprising:means for changing a gain of the image signal, and wherein said meansfor generating electronic mark data further generates electronic markdata for the gain based on time information of a change point of thegain.
 31. The image sensing apparatus as set forth in claim 20, furthercomprising: means for selecting at least one of optical filters thatrestrain or compensate incident light that enters said means forsensing, wherein said means for generating electronic mark data furthergenerates electronic mark data for the optical filter based on timeinformation of a selection point of the optical filter selected by saidmeans for selecting.
 32. The image sensing apparatus as set forth inclaim 20, wherein said means for generating electronic mark datapackages electronic mark data generated after said means for sensingissues a sensing start command until said means for sensing sensingissues a sensing end command.
 33. The image sensing apparatus as setforth in claim 20, further comprising: means for recording the imagesignal to a recording medium, wherein the electronic mark data generatedby said means for generating electronic mark data is recorded to therecording medium along with the image signal.
 34. The image sensingapparatus as set forth in claim 20, further comprising: recording meansfor recording the image signal to a recording medium, wherein theelectronic mark data generated by said means for generating electronicmark data is recorded or stored to a recording medium or a storingmedium different from the recording medium for the image signal.